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Abstract 

This paper demonstrates the capability to obtain and dynamically 
track eigenvalues and eigenvectors using continuous optimization on 
constraint manifolds. 

Introduction 


The author has demonstrated [1] the capability of continuous 
optimization on constraint manifolds (COCM) [2-7] to obtain both 
numerical and closed form analytical solutions to optimization 
problems with C k constraints and objective function. This paper 
displays another useful capability of COCM, that of deriving 
algorithm structures. The ability to formulate the 
eigenvalue/eigenvector problem as an optimization problem leads to 
algorithm structures for obtaining the eigenvalues and eigenvectors 
of a matrix A 

To be consistent with the illuminating tensor notation of Gerretsen 
[8], row vector notation is used throughout this paper. Thus, a vector 
v with components a j in basis {b -) , ... ,b n } may be expressed as 
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v = a B 


where 

a= [ai-..a n ] and B = 


All computer solutions were performed on a Macintosh Plus ™ using 
Lightspeed Pascal ™. 

Summary of COCM 

The following theory is based on concepts from differentiable 
manifolds and differential geometry as described by Thorpe [9] and 
Boothby [10]. 

An n-dimensional manifold is a connected, locally compact space 
with a countable basis, each point of which has a neighborhood 
homeomorphic to euclidian n-space. 

A C k differentiable manifold is a manifold with additional 
mathematical properties imposed which permit the definition of 
compatible coordinate systems on the manifold which are mapped by 
diffeomorphic functions from the manifold into euclidian n-space. 

This mapping is depicted by Figure 1. 





Manifold Patch with Mapping into R n 
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M c = { P I g(p) = c } can be shown to be a C 


k 


differentiable manifold. 


Such a manifold may be called a "constraint manifold" since it is 
totally defined given a set of C k constraint functions. 

The nonlinear programming problem may be stated as 


extremalize f(x) 
over x 

subject to g(x) = constant. 

For C k functions f and g with k > 0 and for the Jacobian matrix 


5 X 9 


~d^Q^...d n Q^ 
-5l 9nv--5n9m 


3g j 

where 3^=^., 

there exist tensors N p and T p at the point p defined by 

Np = 5 T p g (5 p g 5 T p g)- 1 5 p g and T p =I-N p 

where 3 T p g is the transpose of 3 p g and the subscript p represents 
evaluation of the functions at p. 

Geometrically, T p projects onto the tangent space to the constraint 
manifold at p, and N p projects onto the normal space to the 
constraint manifold at p. 
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at the Tensor Operating Point p. 

A manifold M c is covered by a vector field if and only if at each point 
p there is a vector v(p). If f(x) is C k function over M c then the 

gradient 3 p f forms a C k vector field over M c . If f(x) and g(x) are C k 
functions over M c then 3 p f T p forms a C k vector field over M c . 

It is important to note that 3 p f T p is the restriction of 3 p f to M c . 
That means that 3 p fT p is in the tangent space to M c with origin at p. 

Tanabe [4] has shown that under appropriate second order conditions, 
the flow 

P = ± V T P 

extremalizes f(x) on M c by converging to a local extremum at steady 
state. 

A flow is a geodesic on a manifold M c if and only if all acceleration 
is normal to the manifold. 

The unit velocity flow 
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±5 P fT p 

P = ; 

Va p fT p a T p f 

on M c is thus a geodesic. 

A trivial extension of Tanabe's results [4] shows that under 
appropriate second order conditions the unit velocity flow 
extremalizes f(x) on M c by converging to a local extremum at steady 
state. 

The Eiaenvalue/Eiaenvector Optimization Problem 

For the remainder of this paper, the subscript denoting the tensor 
operating point will be dropped but it will still be assumed that the 
point exists and is used. 

Let 'P-i ... 'Pp be the eigenvectors of a matrix A with eigenvalues Xjj. 
Then for 


vp = 

Vi 

and diagonal matrix 

A 

...0 


_Vn_ 



_ 0 ... x nn _ 


we have 

*P A = A *P With vp vp T = vp T ip = i 

Note that 

A = *P T A*P= Sii.Wvi 

i 

has components A.,, in the tensor basis matrices \|/j T \|/j. 

The eigenvalues A.,, and eigenvectors \|/j may be found by solving the 
following tensor problem. 

Maximum \|/-| A \)/-| T y 2 A V 2 T \|/ n A v|/ n T 

subject to \|/-| \|/-| T = 1 \|/ 2 Vi T = 0 ■ ■ ■ \j/ n \)/ 1 T = 0 
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¥2 ¥2 T = 1 Vn ¥2 T = 0 

¥n W = 1 


Suppose that we have \j/-| , ... , \|/ k _-| so far, then we first wish to find a 
vj/ k which satisfies \|/ k \|/| T = 0,1 = 1,..., k-1 with \j/ k \|/ k T = 1 . Then we 
wish to find the v|/ k such that \j/ k A \|/ k T is a maximum over all eligible 
v|/ k . To find an eligible \j/ k we first choose a random vector y k and 
then subtract out components o k in the direction of previously found 
eigenvectors. This ensures orthogonality, that is, 

(y k'°k) 6'i T -Vk-i] = 0 - 

We must now find o k . Define 
H k .i = £h T -Vk-i]- 

Thus 

Yk H k _i = o k H k . 1 . 


Let 


Ok = yk H k -t (Hk-, T Hk-,)- 1 H k ., T , 


then 


Ok Hu =y k H k ., (H k ., T H k .,)-' Hk./Hk., = y H k ., 
as desired. Let 

Qk-i = H k-i (H k-i T H k _i ) _1 H k .-| T and P k _i = I - Q k _i 


then 


o k = YkQk-i and y k - = y k - y k Qk-i = Y k Pk-i ■ 


Let 
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Vk - y k Pk-1 ■ 

Therefore 

¥k Hk-i = y k Hk-i - y k Qk-i H k _i = y k H k .-| - y k H k .-| = 0 
as desired. 

The optimization problem becomes 

Max y-i P 0 A P 0 y/ y 2 Pi A Pi y 2 T ■■■ y n P n -i APn-! y n T 

s-t. yi p oyi T =i y 2 Piy 2 T =i ■■■ y n Pn-iy n T =i- 

Each subproblem is of the form 

Max y k P k _-, A P k _-, y k T 
s-t. y k P k .i y k T = 1 • 

for which 

d p 9 = 2 y ! P k _i 

and 

3 p g 3 p g T = 2y ! P k _i P k .-, T y / 2 = 4 y , P k _-, y/ = 4 . 

Thus 

N k = ai p g (a p g 3T p g)-i 3 p g 
= ^ T p9 (4 ) - 1 a p g 
= ^ T P g ^ P g /4 

= (2P k , y k T y k P k .i 2)/4 
= P k-1 yk T yk P k-1 

and 
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Tk = I - N k = I - p k-1 Yk T Yk Pk-1 


Thus 

Yk = dyfk T k 

= 2 Y k P k -1 A P k _-| ( I - P k -1 y k T Yk Pk -1 ) 

= 2 y k Pk-i A P k .i - 2 y k P k _i APy Yk T Yk Pk-i 

Let 

^kk = Y k Pk-i A P k .-| y k T or 
^kk = ¥k A Vk T > 

then 

Yk = 2 y k P k -i A P k _i - 2 X kk y k P k _i . 

For 


Yk =0 
we have 

Yk P k -i A P k .-| = y k P k -i ^kk or 

¥k A P k-i = ¥k 

where X kk and \j/ k are the eigenvalue and eigenvector of A P 
question becomes, is 

¥k A = y k ^kk 

also. Suppose that is not true, then 
Vk A V|/ k ^kk 


k-i- The 
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implies that 

¥k A P k_i \)/k Pk-1 ^kk = y k Pk-1 Pk-1 ^kk = y k Pk-1 ^kk = ¥k ^kk 

which is false. Thus 


¥k A = Vk ^kk ■ 

The Algorithm Structure 

To start the problem we set P 0 = I and choose y 1 such that y 1 P 0 yi T 
= 1, form P 1 from \|/ 1 = y 1 P 0 , choose y 2 such that y 2 P 1 y 2 T = 1 and 
calculate \\r 2 = y 2 Pi - form P 2 from ¥1 > ¥2 and so on until we got ¥n = 
y n P n . With the problem properly initialized we begin the 
simultaneous solution of the n subproblems 

^kk = y k Pk-1 A P k-1 yk T 

yk = 2 y k Pk-i a p k -i - 2 x k k yk Pk-i 

for which >i k k is the current approximation of the kth eigenvalue and 
¥k = y k Pk -1 

is the current approximation of the kth eigenvector. 

The Dynamic Case 

The dynamic capabilities to track the eigenvalues and eigenvectors 
become obvious from the algorithm structure. If A = A(t) and if 


Yk(t) = 0 , 
then at t + 8t if 

A(t + 8t) * A(t) then 

y k (t + St) * 0. 
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This causes the dynamic process to converge toward the new 
eigenvalues and eigenvectors. It does not, however, guarantee that 
the the new eigenvalues and eigenvectors will be attained. The 
practical case sees a tracking lag resulting in a perpetual error until 
A(t) becomes constant. 

Conclusions 


From a computational viewpoint, the simultaneous solution of the 

resulting n 2 components can present a problem. However, this can 
be alleviated memory wise and speed wise somewhat by solving 
first for the largest eigenvalue, then solving for the next largest and 
so on. 

It is most interesting to note, however, that not only can continuous 
optimization on manifolds provide dynamic numerical and closed 
form solutions to optimization problems [1], but can also be used to 
derive closed form dynamic algorithm structures. The author has on 
a number of other occasions applied this cabability of COCM to 
derive dynamical systems corresponding to specific problems and 
then use these systems to obtain a better theoretical understanding 
of the problem. 
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